23 research outputs found
Scaling Configuration of Energy Harvesting Sensors with Reinforcement Learning
With the advent of the Internet of Things (IoT), an increasing number of
energy harvesting methods are being used to supplement or supplant battery
based sensors. Energy harvesting sensors need to be configured according to the
application, hardware, and environmental conditions to maximize their
usefulness. As of today, the configuration of sensors is either manual or
heuristics based, requiring valuable domain expertise. Reinforcement learning
(RL) is a promising approach to automate configuration and efficiently scale
IoT deployments, but it is not yet adopted in practice. We propose solutions to
bridge this gap: reduce the training phase of RL so that nodes are operational
within a short time after deployment and reduce the computational requirements
to scale to large deployments. We focus on configuration of the sampling rate
of indoor solar panel based energy harvesting sensors. We created a simulator
based on 3 months of data collected from 5 sensor nodes subject to different
lighting conditions. Our simulation results show that RL can effectively learn
energy availability patterns and configure the sampling rate of the sensor
nodes to maximize the sensing data while ensuring that energy storage is not
depleted. The nodes can be operational within the first day by using our
methods. We show that it is possible to reduce the number of RL policies by
using a single policy for nodes that share similar lighting conditions.Comment: 7 pages, 5 figure
Eagle: End-to-end Deep Reinforcement Learning based Autonomous Control of PTZ Cameras
Existing approaches for autonomous control of pan-tilt-zoom (PTZ) cameras use
multiple stages where object detection and localization are performed
separately from the control of the PTZ mechanisms. These approaches require
manual labels and suffer from performance bottlenecks due to error propagation
across the multi-stage flow of information. The large size of object detection
neural networks also makes prior solutions infeasible for real-time deployment
in resource-constrained devices. We present an end-to-end deep reinforcement
learning (RL) solution called Eagle to train a neural network policy that
directly takes images as input to control the PTZ camera. Training
reinforcement learning is cumbersome in the real world due to labeling effort,
runtime environment stochasticity, and fragile experimental setups. We
introduce a photo-realistic simulation framework for training and evaluation of
PTZ camera control policies. Eagle achieves superior camera control performance
by maintaining the object of interest close to the center of captured images at
high resolution and has up to 17% more tracking duration than the
state-of-the-art. Eagle policies are lightweight (90x fewer parameters than
Yolo5s) and can run on embedded camera platforms such as Raspberry PI (33 FPS)
and Jetson Nano (38 FPS), facilitating real-time PTZ tracking for
resource-constrained environments. With domain randomization, Eagle policies
trained in our simulator can be transferred directly to real-world scenarios.Comment: 20 pages, IoTD
Pible: Battery-Free Mote for Perpetual Indoor BLE Applications
Smart building applications require a large-scale deployment of sensors
distributed across the environment. Recent innovations in smart environments
are driven by wireless networked sensors as they are easy to deploy. However,
replacing these batteries at scale is a non-trivial, labor-intensive task.
Energy harvesting has emerged as a potential solution to avoid battery
replacement but requires compromises such as application specific design,
simplified communication protocol or reduced quality of service. We explore the
design space of battery-free sensor nodes using commercial off the shelf
components, and present Pible: a Perpetual Indoor BLE sensor node that
leverages ambient light and can support numerous smart building applications.
We analyze node-lifetime, quality of service and light availability trade-offs
and present a predictive algorithm that adapts to changing lighting conditions
to maximize node lifetime and application quality of service. Using a 20 node,
15-day deployment in a real building under varying lighting conditions, we show
feasible applications that can be implemented using Pible and the boundary
conditions under which they can fail.Comment: 4 pages, 4 figures, BuildSys '18: Conference on Systems for Built
Environments, November 7--8, 2018, Shenzen, Chin
B2RL: An open-source Dataset for Building Batch Reinforcement Learning
Batch reinforcement learning (BRL) is an emerging research area in the RL
community. It learns exclusively from static datasets (i.e. replay buffers)
without interaction with the environment. In the offline settings, existing
replay experiences are used as prior knowledge for BRL models to find the
optimal policy. Thus, generating replay buffers is crucial for BRL model
benchmark. In our B2RL (Building Batch RL) dataset, we collected real-world
data from our building management systems, as well as buffers generated by
several behavioral policies in simulation environments. We believe it could
help building experts on BRL research. To the best of our knowledge, we are the
first to open-source building datasets for the purpose of BRL learning